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PREFACE 


Under  the  Personnel  and  Readiness  Task  (T-5-353)  sponsored  by  the  Office  of  the 
Secretary  of  Defense  (Program  Analysis  and  Evaluation),  the  Institute  for  Defense 
Analyses  (IDA)  studied  the  possibility  of  improving  the  way  in  which  policies  relating  to 
manpower,  personnel  and  training(MPT)  are  developed  and  evaluated- with  principal 
attention  given  to  the  Navy.  A  concise  review  and  digest  of  existing  literature  relating 
personnel  characteristics  and  MPT  policies  to  unit  readiness  and  performance  was 
provided.  It  was  concluded  that  MPT  policies  should  be  more  explicitly  designed  to 
improve  the  performance  of  individuals  and  units  in  the  military,  that  there  is  adequate  data 
to  quantitatively  relate  MPT  factors  to  performance,  and  that  a  research  program  to  develop 
such  quantitative  relationships  should  be  undertaken. 

The  author  is  grateful  for  the  many  helpful  suggestions  received  from  Dr.  Richard 
S.  Elster,  Deputy  Assistant  Secretary  of  the  Navy  (Manpower),  Mr.  Douglas  May,  his  staff 
director,  and  Mr.  Mark  Mohler  of  OSD  (Program  Analysis  and  Evaluation).  The  assistance 
of  Dr.  Robert  M.  Carroll  and  Ll  Cdr.  Robert  C.  Carter  of  the  Office  of  the  Deputy  Chief  of 
Naval  Operations  (MPT)  i:  clso  gratefully  acknowledged. 


EVALUATING  NAVY  MANPOWER,  PERSONNEL  AND 
TRAINING  POLICIES  IN  TERMS  OF  PERFORMANCE 


A.  INTRODUCTION 

In  this  paper  a  new  approach  is  developed  to  evaluate  policy  options  in  the  area  of 
manpower,  personnel  and  training  (MPT).  The  approach  is  based  on  a  belief  that  wiser 
MPT  policy  will  result  if  policy  decisions  are  assessed  in  terms  of  their  implications  for  the 
achievement  of  broad  national  defense  goals.  After  additional  background  is  presented,  the 
paper  develops  a  causal  chain,  or  framework,  that  links  MPT  decisions  to  broad  measures 
of  defense  capability.  Then  the  feasibility  of  transforming  the  conceptual  chain  into  a  set  of 
quantitative  planning  tools  is  considered,  and  a  program  for  developing  such  tools  is 
recommended. 

Decisions  about  MPT  policies  or  the  allocation  of  MPT  resources  are  rarely  made 
with  explicit  reference  to  their  quantitative  contribution  to  national  defense  capability.  This 
is  not  surprising.  There  is  far  too  much  uncertainty  both  in  international  politics  and  in 
warfare  for  predictions  to  be  made  with  any  degree  of  confidence  regarding  the  impact  of 
MPT  decisions  within  the  Defense  Department  on  national  security.  The  real  question  to  be 
considered  by  MPT  policy-makers  is  whether  they  can  be  evaluated  using  indicators  having 
a  stronger  apparent  relationship  to  that  ultimate  measure  than  the  ones  being  used  today. 
Linking  MPT  policy-making  with  measures  of  performance  would  put  DOD  in  a  position  to 
receive  better  performance  for  its  money.  Four  "reas  of  policy  that  could  benefit 
substantially  from  such  linkages  are  requirements  determination,  compensation,  retirement 
and  training. 

The  statements  of  the  military  services  describing  their  personnel  requirements 
shape  the  context  in  which  MPT  decisions  arc  made.  The  mixes  of  experience  and  mental 
ability  that  the  services  say  are  needed  are  derived  w  ^ '  at  reference  to  how  well  other 
mixes  would  perform  or  to  how  much  they  would  cost.  The  compensation  system  then 
strives  to  keep  the  requirements  filled.  For  example,  consideration  is  not  given  to  the 
possibility  that  senior  people  contribute  so  much  to  military  performance  in  some  settings 
that  money  could  be  saved,  military  capability  improved  and  training  costs  reduced  by 
paying  these  people  more  to  keep  greater  numbers  of  them  in  uniform,  while  bringing  in 
fewer  new  recruits.  Under  the  current  system,  the  thinking  is  that  this  action  would  just 
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lead  to  overfilling  requirements  for  senior  personnel  and  that  some  would  have  to  be  forced 
to  leave. 

Similarly,  today's  retirement  system  is  reasonably  efficient  for  achieving  the  level 
of  post-20-year  personnel  implied  by  the  services’  statements  of  requirements.  However, 
an  examination  of  units  with  more  men  of  "post-retirement''  age  might  show  this  level  to  be 
based  on  misguided  assumptions.  A  performance-oriented  approach  to  requirements  could 
suggest  modifications  to  the  retirement  system  that  would  allow  it  to  serve  better  the  real 
needs  of  the  services. 

Considerable  effort  and  expense  go  into  the  design  and  execution  of  programs  of 
training  in  the  military.  Individual  training  programs  are  usually  evaluated  on  the  basis  of 
individual  test  results  after  training,  with  some  feedback  from  the  field.  Further  follow-up 
that  examined  the  performance  of  units  manned  by  individuals  trained  in  different  ways 
could  give  vital  clues  to  people  interested  in  designing  effective  training.  Unit  training 
programs  could  also  benefit  from  such  examination. 

Tying  MPT  policies  to  performance  would  not  only  facilitate  the  adoption  of  better 
MPT  policies,  but  might  also  strengthen  the  position  of  MFT  advocates  when  they  compete 
against  advocates  of  additional  forces  and  modernization  for  increasingly  scarce  funds. 
When  manpower,  personnel  and  training  resources  can  be  linked  to  warfighting  capability, 
their  need  can  be  more  easily  justified.  This  should  lead  to  greater  success  in  negotiations 
for  budget  dollars. 

While  much  of  the  discussion  in  this  paper  focuses  on  the  Navy,  examples  from  all 
the  services  are  used.  The  principles  addressed  here  are  equally  relevant  throughout  the 
DOD. 

B.  FRAMEWORK  DEVELOPMENT 

In  order  to  assure  that  this  paper  is  focused  on  the  real  output  of  the  defense 
decision-making  process—warfighting  capability-framework  development  begins  by 
considering  the  prerequisite  factors  for  combat  success.  The  approach  taken  is  to  move 
from  the  abstract  to  the  more  concrete  one  step  at  a  time.  At  every  step  the  question  is 
asked,  "What  do  we  need  to  get  that?"  By  the  end  of  the  process  the  categories  of 
resources  that  MPT  policies  are  concerned  with  are  reached.  Then,  retracing  these  steps, 


the  links  that  must  be  forged  to  connect  MPT  policies  with  combat  effectiveness  are  seen. 
Incremental  steps  that  can  be  taken  to  make  MPT  decision  making  more  output  oriented, 
even  if  the  most  difficult  links  remain  unforged,  are  also  evident 

1.  The  Determinants  of  Combat  Capability 

Figure  1  illustrates  the  most  commonly  used  taxonomy  for  describing  the 
components  of  military  capability,  the  four  pillars.  This  structure  illustrates  the  fact  that 
successful  pursuit  of  military  goals  requires  four  inputs:  force  size,  modernization 
(reflecting  the  inherent  level  of  capability  of  individual  units),  readiness,  and  sustainability. 
While  this  taxonomy  is  correct  as  far  as  it  goes,  it  masks  the  distinction  between  factors  that 
affect  the  capability  of  units  and  those  that  permit  a  unit  with  a  particular  level  of  capability 
to  fight  effectively.  A  modification  may  more  clearly  reflect  the  way  in  which  factors 
interact  to  produce  overall  defense  capability. 


What  components  are  needed  for  a  force  with  the  requisite  military  capability?  They 
are  the  right  combination  of  force  size  and  unit  capability  to  perform  the  operation  and  the 
ability  to  sustain  it  as  long  as  is  necessary.  Of  course,  the  ability  to  do  the  job  depends  on 
what  the  job  is.  That  is,  it  depends  on  the  threat  being  faced  and  on  the  scenario-what 
both  sides  are  trying  to  accomplish. 

As  Figure  2  shows,  the  capability  of  a  unit  is  determined  by  a  combination  of  its 
design  capability  and  its  readiness.  This  is  consistent  with  the  standard  definition  of 


readiness  as  the  ability  of  a  force,  unit,  weapon  system  or  equipment  to  perform  the 
mission  for  which  it  was  organized  or  designed.  Since  MPT  policy  affects  defense 
capability  largely  through  its  influence  on  readiness,  the  discussion  will  be  confined  to  the 
underpinnings  of  readiness,  although  links  between  various  categories  of  expenditures  and 
the  other  components  of  defense  capability  could  also  be  developed. 


2.  Categorizations  of.  Readiness 

The  standard  treatment  of  readiness,  as  set  out  in  JCS  Pub.  I  and  institutionalized 
in  the  Unit  Status  and  Identity  Reporting  (UNITREP)  system,  breaks  down  overall 
readiness  into  four  components:  material  readiness,  personnel  readiness,  training 
readiness,  and  supply  readiness  (principally  ordnance  for  the  Navy).  This  is  illustrated  in 
Figure  3.  Even  more  than  was  the  case  for  the  four  pillars  of  defense  capability,  this 
taxonomy  masks  complex  interactions  among  the  components  of  readiness. 

It  perpetuates  the  incorrect  notion  that  there  are  four  dimensions  of  readiness  that  do 
not  affect  each  other.  The  concept  that  having  better  people  can  cause  greater  material 
readiness  is  not  formally  admitted  to  the  structure,  for  example.  Neither  is  the  possibility 
that  unit  training  can  enhance  the  performance  of  individual  tasks.  Absence  of  such  links 
severely  hampers  the  value  of  the  UNITREP  structure  as  a  framework  to  guide  analysis. 
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Figure  3.  THE  OFFICIAL  VIEW  OF  READINESS-FOUR  MORE  PILLARS 


What  docs  a  unit  need  to  be  ready?  It  needs  equipment  that  works;  it  needs  a  crew 
capable  of  using  the  equipment  effectively;  and,  it  needs  something  to  shoot,  if  its  mission 
involves  shooting.  That  is  to  say,  as  Figure  4  shows,  that  the  three  components  of  overall 
unit  readiness  are  training  readiness,  material  readiness  and  supply  readiness.  Does  that 
mean  that  personnel  readiness  plays  no  role?  Not  at  all.  It  plays  a  key  role,  but  the  role  is 
multifaceted  and  indirect 


Figure  4.  A  MODIFIED  VIEW  OF  READINESS 


3.  Perspectives  on  Personnel  Readiness 


Personnel  readiness  is  determined  by  the  quality  and  quantity  of  people  attached  to  a 
unit.  Personnel  quality  is  best  thought  of  as  being  largely  synonymous  with  the  level  of 
individual  performance.  Its  relationship  to  the  sorts  of  measures  that  are  usually  used  to 
gauge  personnel  quality  will  be  discussed  below.  It  is  not  determined  solely  by  the  skills 
individuals  had  before  arriving  at  a  unit  The  accomplishment  of  unit  training  can  hone  the 
performance  of  individual  tasks  at  the  same  time  it  improves  the  performance  of  team  tasks. 
The  level  of  unit  training,  in  turn,  depends  not  only  on  how  much  unit  training  is  done,  but 
also  on  how  much  is  needed-a  function  of  the  amount  of  personnel  turnover  and  the  ability 
of  individuals  to  absorb  their  team  tasks  (as  well  as  the  degree  to  which  skills  are 
perishable).  This  ability  presumably  depends  on  personnel  quality. 

As  Figure  5  shows,  personnel  readiness  is  part  of  a  complex  web  of 
interrelationships  that  underlies  material  and  training  readiness.  Material  readiness, 
essentially  the  availability  of  equipment  to  perform  when  called  on,  is  determined  by  how 
often  the  equipment  breaks  (its  reliability)  and  how  long  it  takes  to  repair  when  it  is  down. 
Reliability  is  usually  taken  to  depend  only  on  equipment  design  and  the  environment  in 
which  it  is  used.  In  fact,  there  are  undoubtedly  user-induced  failures,  but  they  will  not 
be  incorporated  explicitly  into  the  framework  being  developed  here.  The  speed  of 
repair,  while  dependent  on  the  availability  of  parts,  is  also  highly  dependent  on  the  ability 
and  availability  of  maintenance  personnel--that  is,  on  their  personnel  readiness.  It  should 
also  be  remembered  that  maintenance  is  not  strictly  an  individual  job.  Trouble  shooting  is 
to  a  considerable  extent  a  team  effort  One  would  expect  the  speed  of  repair  to  depend  not 
only  on  the  quality  and  quantity  of  repair  personnel,  but  also  on  their  level  of  unit  training. 

Unit  training  is,  of  course,  also  expected  to  be  a  key  factor  underlying  training 
readiness— that  is,  the  ability  of  units  to  use  their  equipment  effectively  when  it  is  working. 
It  is  reasonable  to  believe,  however,  that  there  is  a  trade-off  between  personnel  readiness 
and  the  unit  training  level  in  the  production  of  training  readiness.  Thus,  it  should  be 
possible  to  achieve  a  given  level  of  operating  proficiency  with  fewer  team  skills  if 
personnel  readiness-the  quality  and  quantity  of  personnel— is  higher. 

Thus  far,  the  discussion  in  this  paper  has  been  quite  abstract.  It  has  not 
incorporated  the  kinds  of  measures  of  personnel  quality  that  are  usually  used  in  judging  the 
success  of  MPT  pol'cies.  It  is  now  time  to  turn  to  that  incorporation. 
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Figure  5.  THE  ROLE  OF  PERSONNEL  ANO  UNIT  TRAINING  IN  PRODUCING 
UNIT  READINESS 


Discussions  of  the  quality  of  the  Navy's  enlisted  force  often  revolve  around  three 
factors:  the  distribution  of  personnel  by  mental  group,  as  measured  by  entry  test  score; 
their  education  level  (the  fraction  who  graduated  from  high  school);  and  the  length  of  time 
they  have  been  in  the  Navy  (or  their  pay  grade  distribution).  It  is  reasonable  to  assume  that 
these  factors,  along  with  the  amount  and  type  of  Navy  schoolhouse  training  they  have 
received  are  key  determinants  of  individual  performance.  Motivation  also  plays  a  central 
role,  as  Figure  6  reflects.  Unfortunately  both  the  level  of  motivation,  and  the  influence  of 
policies  on  motivation  are  very  hard  to  quantify. 

Similarly,  unit  training  accomplishment  is  often  considered  to  be  directly  related  to 
the  quantity  of  steaming  (or  flying)  time  available  to  units.  Actually  the  situation  is  more 
complex.  The  training  syllabi  that  units  strive  to  complete  do  require  operating  time,  but 
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not  all  steaming  time  (or  flying  time)  is  the  same  for  the  purposes  of  producing  training. 
As  an  example,  steaming  time  can  be  broken  down  into  transit  time,  operations  time, 
project  support  time,  explicit  training  time,  and  other  steaming.  Not  all  steaming  time  is 
labeled  as  training  time,  and  not  all  underway  training  gets  done  during  "training"  time. 
Further  disaggregation  of  steaming  categories  is  possible,  and  probably  would  be  fruitful 
for  consideration  of  the  determinants  of  personnel  performance.  But  steaming  (or  flying) 
isn't  the  only  way  to  get  unit  training.  Non-underway  training  can  be  accumulated  in 
trainers  or  simulators.  Also,  some  parts  of  the  training  syllabi  can  be  completed  aboard 
ship  while  in  port 


The  development  of  our  framework  has  now  reached  some  familiar  ground.  The 
factors  on  the  left  side  of  Figure  6  are  things  the  Navy  is  used  to  thinking  about  buying  or 
influencing  with  MPT  resource  decisions.  The  next  section  addresses  the  link  to  resources. 
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5.  Manpower.  Personnel  and  Training  Resources  for  Readiness 


Figure  7  depicts  the  plethora  of  relationships  between  various  categories  of 
resources  and  the  factors  underlying  personnel  performance  discussed  above.  Some  of  the 
relationships  are  straightforward:  trainers  make  non-underway  training  time  available,  fuel 
is  necessary  for  increased  underway  time,  Navy  schools  must  have  funding  if  individual 
training  is  to  be  increased.  The  resources  associated  with  accession  and  compensation, 
however,  can  be  expected  to  (and  often  have  been  shown  to)  have  multidimensional 
outputs. 


External  Factors 


Unit 

Personnel  Stability 


Non-Underway 
Training  Time 


Underway  Time 
(by  category) 

Entry  Test  Score* 

Motivation 

Educational  Level 

Navy  Experience 


#  Indvidual  Navy 
Training 


Personnel 

Quantity 


Figure  7.  LINKS  TO  RESOURCES 


'thus,  for  example,  more  recruiters  and  advertising  will  raise  the  mental  group 
distribution  of  new  recruits  and  attract  more  high  school  graduates.  So  will  pay,  benefits 
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and  tea  pay.  By  encouraging  extensions  at  sea,  tea  pay  Mould  alto  improve  the  level  of 
unit  training  by  increasing  crew  stability.  These  compensation  tools  also  affect  retention 
(though  some  benefits,  like  the  G.I.  Bill,  can  be  expected  to  decrease  the  level  of 
experience  in  the  Navy  by  decreasing  retention).  Reenlistment  bonuses  and  quality -of-life 
expenditures  are,  of  course,  expected  to  improve  experience  levels  by  improving  retention. 

They  may  also  be  particularly  important  in  retaining  individuals  with  high  entry  test 
scores  and  education  levels.  The  retirement  system  obviously  plays  a  central  role  in 
shaping  the  experience  profile  of  the  Navy  personnel  structure.  In  addition,  both  the 
monetary  and  quality-of-life  aspects  of  Navy  life  can  affect  the  motivation  of  personnel. 
Also,  by  influencing  service -wide  turnover,  they  help  determine  the  personnel  stability  of 
individual  units. 

Decisions  to  increase  the  number  of  personnel  in  the  Navy,  holding  constant  levels 
of  personnel  quality,  require  additional  expenditures  in  most  of  the  MPT  resource 
categories. 

Of  course,  neither  the  quantity  nor  the  quality  of  personnel  is  determined  solely  by 
DOD  and  service  policy.  It  depends  critically  on  the  national  environment.  Fewer  recruits 
will  be  attracted  when  entry-age  cohorts  are  small,  and  more  will  be  when  civilian  job 
opportunities  arc  poor.  Such  external  factors  should  have  (and  have  had)  an  explicit  place 
in  MPT  policy  analysis. 

While  not  all  of  the  links  hypothesized  in  Figure  7  have  been  quantified,  many  of 
them  have  been.  The  challenge  facing  MPT  policy  makers  and  researchers  is  to 
quantitatively  connect  the  factors  on  the  right  side  of  Figure  7  with  measures  of 
performance,  readiness  and  capability  that  are  further  along  the  chain  of  causation  that  runs 
from  the  size  of  the  MPT  budget  to  the  production  of  overall  defense  capability, 

6.  Applicability,  of  tta  Framework 

Thus  far,  this  paper  hat  presented  a  series  of  relationships  that  fit  together  into  an 
overall  framework  connecting  MPT  resource  and  policy  inputs  to  a  hierarchy  of  outputs  of 
the  defense  management  process.  This  framework  is  presented  in  Figure  8.  While  the  web 
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of  interrelatioiships  appears  somewhat  daunting,  it  suggests  a  methodology  for  evaluating 
MPT  policies  in  broader  terms  than  are  typically  considered.1 

This  kind  of  evaluation  requires  three  things:  an  ability  to  develop  quantitative 
indicators  of  some  of  the  concepts,  such  as  individual  performance  or  training  readiness, 
that  should  be  measured;  an  ability  to  match  the  characteristics  of  personnel  with  their 
performance  (or  their  unit's  readiness);  and  the  resolve  and  resources  to  do  the  research  to 
quantify  the  hypothesized  links.  Before  a  research  effort  is  embarked  on,  however,  it  is 
important  to  establish  that  adequate  data  exists  to  support  the  effort  It  is  the  question  of 
feasibility  that  must  be  raised  as  the  next  step  in  considering  a  program  to  give  MPT 
policies  their  due  as  crucial  determinants  of  the  strength  of  the  Navy. 

C.  INDICATORS  OF  PERFORMANCE 

Forming  and  evaluating  personnel  policies  on  the  basis  of  the  performance  of 
personnel  and  units  can  only  be  done  by  using  information  on  such  performance.  An 
analysis  of  what  kinds  of  people  tend  to  perform  best  under  what  kinds  of  circumstances 
cannot  be  made  without  accepted  yardsticks  with  which  to  measure  their  performance. 
This  requirement  does  much  to  frustrate  consideration  of  performance-oriented  policy 
making.  For  most  of  the  jobs  performed  in  the  Navy,  policy-makers  do  not  have  routine 
access  to  the  kind  of  information  that  would  allow  them  to  base  decisions  on  how  well 
people  do,  rather  than  solely  on  what  kinds  of  characteristics  they  have.  The  problem  is 
not  that  useful  information  does  not  exist  The  problem  is  that  it  is  usually  not  available  for 
policy  analysis  and  development.  In  this  section  the  purpose  is  to  demonstrate  that 
performance  data  are  generated  for  a  wide  range  of  units,  missions  and  tasks.  Further, 
these  data  are  often  used  to  judge  the  performance  of  people  and  units  in  an  operational 
setting. 

In  terms  of  the  framework  developed  in  the  previous  section,  existing  performance 
indicators  sometimes  approximate  measures  of  unit  capability.  Other  times  they  provide 
information  more  consistent  with  the  notions  of  unit  readiness,  training  readiness,  material 

1The  framework  focuses  on  the  MPT  area.  It  could  be  expanded  to  address  the  contribution  of  all  kinds  of 
resources  to  military  performance.  This  would  facilitate  cost  effectiveness  analyses  that  cut  across  the 
pillars  of  defense  capability  of  the  kind  performed  in  [t]. 


readiness,  or  personnel  quality  as  indicated  by  individual  performance.  In  no  cases  are 
they  perfect  It  is  always  possible  to  develop  arguments  against  their  use  for  policy 
evaluation  and  development  Their  flaws  should  be  borne  in  mind,  but  the  decision  about 
whether  to  use  them  should  depend  not  only  on  how  they  depart  from  the  ideal,  but  also  on 
the  alternative.  What  will  be  used  to  form  MPT  policies  and  justify  MPT  expenditures  if 
existing  indicators  of  performance  are  not?  The  same  old  input  measures --entry  test  scores, 
pre-service  education,  pay  grade  mix,  steaming  days,  and  flying  hours-that  have  a  more 
obvious  tie  to  cost  than  to  effectiveness  will  continue  to  have  the  field  to  themselves.  The 
desire  for  perfection  should  not  be  allowed  to  overshadow  the  possibility  of  substantial 
improvement. 

Enumeration  of  specific  sources  of  potential  performance  indicators  for  the  Navy 
follows.  Measures  oriented  toward  operational  performance  will  be  addressed  first,  and 
then  other  measures  oriented  towards  material  readiness  and  individual  performance.  At 
the  end  of  this  section  sources  of  similar  and,  in  some  cases  better,  information  for  the 
other  services  are  identified. 

1.  Measures  of  Operational  Performance 

a.  Training  inspections.  These  are,  perhaps,  the  best  source  of  information  on 
the  performance  of  units  in  a  combat-like  situation.  They  are  scheduled,  observed,  and 
scored  by  authorities  outside  the  units  involved,  but  the  results  are  not  usually  available 
centrally.  They  are  kept  at  the  type-command  level.  Some  training  inspections,  like 
Operational  Propulsion  Plant  Examinations  (OPPEs)  and  Ship  ASW  Readiness 
Effectiveness  Measuring  Exercises  (SHAREMs)  reflect  proficiency  in  individual  primary 
missions  (in  these  cases  mobility  and  ASW).  Others,  Operational  Readiness  Evaluations 
(OREs),  for  example,  are  broader  assessments  of  a  unit's  performance. 

Of  course,  training  inspections  have  their  drawbacks  as  performance  indicators. 
They  do  not  have  the  realism  of  major  exercises,  but  an  attempt  is  made  to  score  them  in  a 
more  consistent  fashion.  They  are  prepared  for,  and  thus  probably  do  not  reflect  typical 
performance.  Perhaps  they  approximate  an  upper  bound  on  performance.  They  may  only 
occur  at  set  places  in  the  deployment  cycle.  OREs,  for  example,  take  place  shortly  before 
the  start  of  overseas  deployments.  Nonetheless,  if  we  are  willing  to  accept  the  assumption 
that  units  which  do  well  in  training  inspections  are  more  likely  to  perform  well  in  combat, 
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an  assumption  on  which  the  inspections  rest  in  the  first  place,  they  could  provide  a  very 
useful  benchmark  of  the  success  of  MPT  policies. 

b.  Selected  Exercises.  These  are  evolutions  performed  by  ships  as  part  of 
their  regular  training  syllabi.  They  differ  from  other  elements  of  the  syllabi  in  that  they 
must  be  graded  by  outside  observers  once  during  each  cycle  of  competition  for  excellence 
awards.  They  include  live-firing  exercises,  like  naval  gunfire  support  exercises.  Taken 
together,  they  cover  all  of  a  ship's  primary  missions.  Their  weaknesses  are  that, 
individually,  each  only  reflects  a  small  part  of  overall  performance  for  any  particular 
mission,  that  failing  grades  may  not  be  numerically  recorded,  and  that  considerations  other 
than  performance  may  sometimes  play  a  role  in  the  determination  of  a  grade.  Still,  they  are 
a  principal  factor  in  determining  the  recipients  of  awards  in  which  the  Navy  places 
considerable  stock. 

c.  Excellence  Awards.  Scores  on  selected  exercises  are  aggregated  to  help 
determine  the  winners  of  the  mission-area-excellence  awards  held  by  squadron 
commanders.  Multiple  awards  are  usually  given.  Either  the  aggregate  mission-area  scores 
or  the  receipt  of  mission-area  awards  could  be  the  basis  for  the  kinds  of  performance 
indicators  being  studied.  The  overall  Battle-E  competition  may  not  yield  as  suitable  an 
indicator  of  performance.  Only  one  ship  in  a  squadron  can  receive  the  Battle-E,  so  receipt 
of  the  award  may  not  do  much  to  distinguish  good  performers  from  bad  ones.  More 
important,  the  relationships  between  MPT  policies  and  performance  presumably  differ  by 
rating.  This  means  that  performance  measures  that  can  be  associated  with  people  in 
particular  ratings  (or  groups  of  ratings)  will  be  more  helpful  to  this  analysis  than  broader 
measures. 

d.  ftombing  Scores.  Most  of  the  discussion  above  deals  with  information  on 
the  performance  of  ships  (though  air  wings  undergo  OREs).  Sources  of  information  on  the 
operational  performance  of  aircraft  and  aviation  units  will  be  addressed  next.  Attack 
squadrons  keep  track  of  the  accuracy  of  their  practice  bombing  runs.  Different  kinds  of 
bombing  runs  are  identified.  It  is  possible  to  associate  bombing  runs  with  particular  air 
crews,  opening  the  possibility  of  analysis  at  the  individual  (as  well  as  the  unit)  level.  Data 
on  bombing  accuracy  is  forwarded  to  functional  wing  headquarters  and  is  used  to  compare 
the  performance  of  squadrons. 


e.  Air  Combat  Maneuvering  Ranges.  The  Tactical  Aircrew  Combat  System 
(TACTS)  is  the  most  advanced  and  realistic  instrumentation  system  ever  developed  for 
training  air  crews.  It  provides  information,  instantaneously  and  on  tape,  on  the  flight 
dynamics,  weapons  system  status,  and  weapons  firing  of  each  aircraft  engaged  in  a  training 
mission.  Systems  are  installed  at  Yuma  and  at  Oceana  This  information  is  used  to  debrief 
air  crews  about  their  missions.  It  should  be  possible  to  develop  summary  measures  of 
mission  performance  and  relate  diem  to  training  history  (among  other  things).  TACTS  data 
are  held  extremely  closely  and  is  not  now  used  for  the  sort  of  analytic  purposes  being 
considered  here. 

f.  Simulators.  Information  on  the  operational  performance  of  units  and 
personnel  can  be  supplemented  by  data  on  their  proficiency  using  advanced  training 
devices.  The  results  of  simulator  exercise  results  for  P-3  squadrons  are  particularly 
promising  as  a  source  of  information.  Crew  members  on  these  ASW  aircraft,  both  officer 
and  enlisted,  spend  a  substantial  amount  of  time  in  simulators.  These  sessions,  which  last 
several  hours,  are  used  both  as  training  exercises  and  as  indicators  of  crew  readiness.  The 
exercises  are  taped,  and  individual  crew  members,  as  well  as  the  crew  as  a  whole,  are 
graded  on  their  performance.  The  results  of  actual  operational  flights  are  similarly  recorded 
but  they  may  not  provide  a  data  base  as  extensive  or  as  well  controlled.  Training  by  ship 
teams  in  ASW,  AAW  and  EW  training  simulators  ashore  should  also  provide  measures  of 
performance. 

2.  Measures  of  Maintenance  Performance 

While  operational  performance  is  what  the  Navy  ultimately  cares  about,  the 
indicators  of  operational  performance  that  can  be  generated  in  peacetime  largely  reflect  the 
efforts  of  operating  personnel.  They  are  usually  the  result  of  using  equipment  that  has  a 
much  higher  than  typical  probability  of  working  properly.  A  large  proportion  of  Navy 
personnel,  however,  devote  themselves  to  making  equipment  work  properly.  Performance 
measures  that  reflect  the  success  of  their  efforts  are  needed.  Unlike  indicators  of 
operational  performance,  indicators  of  maintenance  performance  typically  are  available 
centrally,  and,  indeed,  are  used  to  judge  trends  in  the  readiness  of  the  fleet.  They  are  not, 
however,  used  very  extensively  to  develop  MPT  policies  calculated  to  enhance  readiness. 


This  section  lists  some  of  the  more  promising  sources  of  information  on  material  readiness 
that  could  be  used  for  that  purpose. 

a.  Casualty  Reports  (CASREPsl.  Whenever  a  ship  suffers  an  equipment 
failure  that  adversely  affects  its  ability  to  cany  out  its  primary  missions  (and  that  cannot  be 
repaired  within  48  hours),  it  is  supposed  to  file  a  casualty  report  CASREPs  range  in 
severity  from  C-2,  which  implies  only  a  partial  degradation  in  mission  performance,  to  C*4 
which  implies  loss  of  the  ability  to  perform  at  least  one  mission.  While  CASREPs  form  the 
basis  of  one  of  the  most  widely  used  indicators  of  fleet  material  condition,  they  have 
serious  shortcomings  as  the  basis  for  analysis  of  the  causes  of  variations  in  material 
readiness  across  ships.  They  are  not  objective,  and  the  criteria  for  filing  CASREPs  may 
vary  widely.  Further,  even  if  they  were  objective,  good  performance  by  a  CASREP-based 
measure  would  not  necessarily  reflect  high  maintenance  performance.  It  might  just  be  that 
the  equipment  was  not  operated  often.  Nonetheless,  CASREPs  have  been  successfully 
used  to  derive  the  measures  of  proficiency  in  at  least  one  pilot  study  of  the  determinants  of 
personnel  performance.  A  virtue  of  CASREP  data  is  that  they  are  taken  seriously  by  type 
commanders  and  by  the  supply  system  for  determining  the  priority  of  requests  for  parts.  In 
addition,  they  are  readily  available  from  the  Fleet  Material  Support  Office. 

b.  The  Unit  Status  and  Identity  Reporting  System.  The  UNITREP 
system  is  the  official  JCS-sponsored  tool  for  measuring  readiness.  It  contains  information 
on  the  status  of  units  in  each  of  four  resource  categories:  personnel,  training,  supply  and 
material.  In  the  first  three  categories  UNITREP  reflects  more  whether  a  unit  has  the 
resources  it  is  supposed  to  have  than  whether  it  can  do  its  job.  In  the  material  readiness 
category,  however,  ships  use  detailed  logic  diagrams  to  assess  their  ability  to  perform  each 
of  their  primary  missions  in  view  of  the  operational  status  of  all  relevant  equipment. 
Unfortunately,  this  mission-specific  information  is  not  forwarded  to  Washington,  and  may 
be  difficult  to  obtain.  Also,  UNITREP  data  suffers  from  some  of  the  same  flaws  as 
CASREP  data:  it  is  self-reported  and  it  is  only  meant  to  capture  the  condition  of 
equipment,  not  the  proficiency  of  the  maintained.  Once  again,  equipment  that  was  never 
turned  on  will  appear  to  have  been  well  maintained. 

c.  Form  4855  Data.  These  data  reflect  information  on  equipment  logs  for 
selected  combat  systems.  It  is  possible  to  tell  when  the  equipment  was  on,  when  failure 
was  detected,  when  parts  were  on  order,  and  when  failures  were  fixed.  They  are  probably 


the  cleanest  ship  maintenance  data  in  the  Navy.  They  are  kept  at  the  Fleet  Analysis  Center 
under  the  sponsorship  of  NAVSEA,  but  it  has  not  always  proved  to  be  accessible. 

d.  Board  of  Inspection  and  Survey  (INSURV1  Data.  The  INSURV 
Board  carefully  inspects  over  100  ships  per  year.  Inspectors  are  highly  trained  and  are 
respected  for  their  objectivity  and  consistency.  The  principal  indicator  of  material  readiness 
generated  by  INSURV  inspections  is  the  Material  Condition  Index  (MCI).  It  disaggregates 
equipment  into  25  categories,  and  assigns  a  score  to  each  category.  For  each,  a  score  of 
zero  indicates  no  deficiency,  a  one  indicates  a  deficiency,  and  a  two,  a  missio..  ^gradation. 
Unfortunately,  the  25  categories  are  probably  not  detailed  enough  to  be  of  use  in  the  kind 
of  analysis  being  considered  here.  For  instance,  there  is  only  one  of  the  25  that  addresses 
all  combat  equipment.  The  INSURV  Board  has  begun  to  develop  a  more  detailed  data  base 
on  selected  individual  equipments.  This  may  be  more  suitable  for  deriving  information  on 
the  maintenance  performance  of  people  in  particular  ratings. 

e.  Aircraft  3-M  Data.  The  Maintenance  and  Material  Management  (3-M) 
System  is  the  Navy's  regular  maintenance  reporting  system.  As  a  source  of  information  on 
the  material  readiness  of  ships,  3-M  data  is  seriously  flawed.  Drawbacks  include  non¬ 
reporting,  reporting  errors,  differences  in  reporting  requirements  from  ship  class  to  ship 
class,  and  changes  in  reporting  requirements  over  time.  In  the  aviation  world,  however,  3- 
M  information  gets  much  more  respect  Perhaps  this  is  because  the  maintenance  action 
forms  used  to  manage  repair  work  are  directly  used  to  feed  the  aviation  3-M  data  base.  In 
any  case,  3-M  data  is  used  to  develop  the  statistics  on  mission  capable  (MC)  and  full- 
mission-capable  (FMC)  rates  that  are  routinely  used  to  track  the  material  readiness  of 
aircraft.  Under  the  Sub-system  Capability  Information  Reporting  (SCER)  system,  aviation 
3-M  data  can  also  be  used  to  generate  more  detailed  information  on  the  readiness  of 
particular  kinds  of  equipment.  This  could  prove  extremely  useful  for  various  MPT 
analyses. 

3.  Measures  of  Individual  Performance 

It  will  not  always  be  possible  to  determine  what  kinds  of  people  and  training  tend  to 
produce  the  best  performance  in  units  in  the  field.  It  will  sometimes  be  impossible  to 
isolate  the  effect  of  MPT  factors  on  operational  and  maintenance  performance.  Some 
enlisted  personnel,  especially  those  in  administrative  and  medical  ratings,  perform  vital 
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functions  but  do  not  much  affect  the  performance  of  units  in  peacetime.  Even  in  such 
cases, better  means  can  be  found  than  basing  MPT  polices  on  the  proxy  measures-such  as 
entry  test  scores— used  today.  Designing  and  validating  policies  in  terms  of  how  well 
people  perform  as  individuals  can  be  tried.  Two  sources  of  readily  available  information 
suggest  themselves. 

a.  Scores  on  Advancement  Examinations.  With  few  exceptions,  in 

to  be  promoted  to  any  pay  grade  above  E-3  in  any  rating  an  individual  must  pass  a  wniu.. 
examination  in  the  skills  that  must  be  mastered  to  perform  successfully  in  that  higher  pay 
grade.  Supervisor  approval-presumably  based  on  actual  performance~is  necessary  to  take 
the  exam.  Of  course,  test  scores  measure  verbal  mastery  of  a  subject,  which  is  not  the 
same  thing  as  mastery  in  the  work  environment  Performance  in  test  taking  is  biased  in 
favor  of  those  who  are  good  at  taking  tests.  St»ll,  the  Navy  obviously  values  these  tests.  It 
only  promotes  people  who  have  passed  them. 

b.  Promotion  Decisions.  Advancement  exams  are  not  the  only  factor  in 
determining  promotion  decisions.  Explicit  consideration  is  given  to  the  supervisor’s 
appraisal  of  non-testable  military  skills.  Whether  or  not  someone  is  promoted  is  probably 
the  best  single  summary  measure  of  how  highly  he  is  regarded  relative  to  other  individuals 
who  are  eligible  for  consideration.  Promotion  results  cannot  be  compared  across  ratings  or 
across  groups  of  people  who  took  advancement  exams  at  different  times  since  the  ease  of 
advancement  depends  on  the  quality  of  the  competition  and  on  the  need  for  people  in  the 
higher  pay  grade.  Also,  promotion  results  do  not  provide  a  continuous  scale— you  are 
cither  promoted  or  you  are  not  How  long  ii  takes  people  who  entered  the  Navy  at  a  given 
time  to  reach  any  particular  pay  grade  docs  provide  such  a  scale,  however.  A  problem  with 
both  advancement  exams  and  promotion  decisions  as  measures  of  individual  performance 
is  that  neither  allows  a  comparison  of  the  value  of  personnel  in  different  pay  grades,  a  key 
element  in  designing  an  efficient  personnel  structure,  since  both  only  address  variations  in 
individual  performance  within  a  single  pay  grade. 

4.  Measures  of  Performance  in  the  Other  Services 

The  Navy  has  a  broad  range  of  performance  measures  that  could  be  used  to  aid  in 
assessing  and  designing  MPT  polices.  The  other  services,  especially  the  Army  and  the 
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Marine  Corps,  have  developed  approaches  to  generating  information  about  performance 
that  might  fruitfully  supplement  the  Navy's.  Some  of  these  will  be  briefly  discussed  here. 

a.  The  Army  Training  and  Evaluation  Program  (ARTEP1.  For  various 
kinds  of  units,  the  ARTEP  provides  a  list  of  mission  elements  and  subelements  that  the 
units  should  be  able  to  perform.  It  is  the  basis  for  unit  training.  The  subelements  have 
standards  that  are  tracked  and  measured  by  units  throughout  training  in  preparation  for 
external  ARTEP  evaluation  The  external  evaluation  provides  objective  information  on 
operational  performance.  Although  ARTEP  evaluation  results  have  sometimes  been  used 
for  analysis  of  policy  issues,  the  Army  has  been  reluctant  to  use  them  systematically. 

b.  The  National  Training  Center  (NTC1.  In  effect,  the  NTC  is  a  large- 
scale  combat  simulator  located  at  Fort  Irwin,  California.  Armor  and  mechanized  units  train 
in  live-fire  exercises  and  in  force-on-force  engagements.  They  are  confronted  by  a  well- 
trained  opposing  force.  Exercises  involve  the  combined  operation  of  tanks,  mechanized 
infantry,  artillery,  air  defense,  engineers,  electronic  warfare,  nuclear,  biological  and 
chemical  warfare,  attack  helicopters,  and  close  air  support  aircraft  They  use  laser-based 
instrumentation  to  assess  casualties  when  a  weapon  fires  and  the  laser  hits  the  target  For 
the  most  part,  data  from  NTC  training  are  used  like  that  from  the  Navy's  TACTS  system, 
to  provide  feedback  to  the  unit  being  trained.  The  Army  recognizes  that  it  is  developing  a 
data  base  with  considerable  potential  to  support  training  research,  as  well  as  research  in 
other  areas. 

c.  Hands-on  Individual  Performance  Data.  The  Army  gives  Skill 
Qualification  Tests  (SQTs)  to  individuals  in  a  wide  range  of  occupational  specialties  and 
skill  levels.  Critical  tasks  within  an  occupational  specialty  are  usually  tested  in  three  ways: 
written  tests;  hands-on  performance;  and  supervisors'  certification  that  the  tasks  have  been 
successfully  performed  during  duty.  In  addition,  the  Systems  Analysis  Activity  of  the 
Army’s  Training  and  Doctrine  Command  (TRASANA)  developed  a  set  of  tests  for  a  variety 
of  military  occupational  specialties  (MOSs).  These  included  both  written  and  hands-on 
components.  The  written  part  of  both  the  TRASANA  tests  and  of  SQTs  are  conceptually 
similar  to  the  Navy's  advancement  exams.  The  hands-on  part  of  the  tests  is  interesting 
because  it  breaks  the  link  between  success  and  the  ability  to  take  written  tests. 


d.  The  Marine  Corps  Combat  Readiness  Evaluation _ System 

(MCCRES).  This  system  divides  the  mission  of  every  kind  of  unit  in  the  Marine  Corps 
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into  myriad  elements.  Over  1000  elements  exist  for  the  Corps  as  a  whole.  Each  element  is 
designed  to  address  a  task  that  is  distinct  enough  to  be  evaluated  on  a  yes-or-no  basis.  The 
purpose  of  the  system  is  to  go  bey  cad  evaluating  whether  a  training  operation  was 
successful  to  identifying  the  strong  and  weak  points  of  execution.  It  is  well  suited  to  help 
experienced  commanders  evaluate  readiness  and  plan  training  programs.  It  could  also  be 
used  as  a  management  tool  for  comparing  units  and  identifying  trends  in  readiness  and 
determinants  of  readiness. 

5.  Using  Data  on  Measures  of  Performance 

It  is  evident  that  measures  of  unit  and  individual  performance  abound  in  the  Navy 
and  in  the  other  services.  It  is  also  evident  that  there  is  a  possibility  of  MPT  policies  being 
substantially  improved  by  systematically  evaluating  them  according  to  their  impact  (or 
expected  impact)  on  such  measures.  There  are,  however,  two  reasons  why  it  might  not  be 
such  a  good  idea  to  embark  on  a  program  of  systematic  evaluation.  First,  using 
performance  measures  in  this  way  could  possibly  negate  their  value  for  the  jobs  they  were 
originally  designed  for.  Second,  it  could  turn  out  to  be  a  waste  of  time.  It  might  just  be 
too  difficult  to  identify  the  effect  of  personnel  characteristics  and  training  experience  on 
measures  of  performance  that  are  influenced  by  many  other  things  as  well.  These  are  both 
serious  objections  and  deserve  discussion. 

The  objection  to  using  performance  data  for  central  management  purposes  seems 
confined  to  information  on  operational  performance.  Information  on  maintenance 
performance  and  individual  performance  is  already  available  centrally,  and,  especially  in  the 
case  of  maintenance  performance,  is  used  to  develop,  assess  and  justify  policy.  There  is 
concern,  though,  that  making  data  on  operational  performance  more  widely  available  would 
dilute  the  incentives  of  commanders  in  the  field  to  evaluate  their  subordinate  units 
accurately  because  of  fear  that  bad  performance  would  reflect  badly  on  them.  Of  course, 
the  value  of  improved  management  information  might  outweigh  this  cost,  but  the  choice  is 
not  clear  cut  A  decision  to  use  measures  of  operational  performance  as  the  basis  for 
analyzing  MPT  polices,  on  the  other  hand,  is  much  clearer.  That  is  because  the  analysis 
can  be  done  without  making  the  performance  data  widely  available.  The  identity  of  the 
units  that  lie  behind  the  data  can  be  suppressed  without  damaging  the  analysis.  The  idea  is 
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not  to  put  units  on  report,  it  is  to  determine  factors  that  are  associated  with  good 
performance. 

But  is  existing  information  on  performance  good  enough  to  use  for  analyzing  MPT 
policies?  The  strongest  possible  evidence  on  this  question  would  be  to  show  that  it  has 
been  used  successfully,  if  sporadically,  to  investigate  a  wide  range  of  MPT  questions  in  the 
Navy  and  in  other  services.  In  the  next  section  such  evidence  will  be  developed. 

D.  ANALYSES  OF  THE  DETERMINANTS  OF  PERFORMANCE 

The  thesis  of  this  paper  is  that  the  Navy  would  be  well  served  by  a  comprehensive 
effort  to  tie  MPT  policies  to  credible  indicators  of  performance  in  the  fleet  Although  no 
such  effort  has  yet  been  begun,  there  have  been  more  than  isolated,  but  less  than  extensive, 
attempts  to  relate  MPT  factors  to  performance  in  the  Navy  and  in  other  services.  These 
attempts  have  been  fairly  successful.  Taken  together,  they  form  a  body  of  literature  that 
engenders  optimism  about  the  feasibility  of  giving  MPT  policy  making  more  of  a 
performance  orientation.  In  this  section  the  purpose  is  to  review  this  literature.  Studies 
that  deal  with  the  determi  rants  of  operational  performance,  maintenance  performance,  and 
individual  performance  are  all  addressed.  Some  relevant  studies  are  undoubtedly  omitted, 
but  the  reader  should  remember  that  the  point  here  is  to  demonstrate  the  feasibility  of  th.s 
kind  of  research,  not  to  exhaustively  survey  the  field. 


Relationships  between  aspects  of  MPT  policy  and  indicators  of  operational 
performance  ha''e  been  developed  for  air,  sea  and  ground  units.  Six  analyses  will  be 
reviewed.  Tnosc  that  deal  with  Navy  units  will  be  discussed  first,  followed  by  Air  Force 
and  then  Army  studies. 

a.  Unit  training  and  the  performance  of  aviation  units.  Recent  work  at 
the  Center  for  Naval  Analyses  has  examined  the  hypothesis  that  aircraft  squadrons  that 
have  had  a  greater  amount  of  recent  flying  time  (and  hence,  presumably,  a  greater  amount 
of  unit  training)  perform  better  [2].  Two  sources  of  performance  information  were  used: 
Operational  Readiness  Evaluations  (OREs)  and  bombing  scores.  The  ORE  results  were 
quite  striking.  Remember  that  OREs  are  tests  of  operational  performance  that  are  graded  by 
observers  from  outside  the  air  wing.  The  tests  are  conducted  shortly  before  deployment  is 
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scheduled  to  take  place.  The  study  looked  at  the  results  of  90  OREs.  It  found  that 
squadrons  that  flew  more  in  the  period  before  the  ORE  had  significantly  higher  and  more 
predictable  boarding  rates  (fewer  aborted  landing  attempts),  significantly  better  landing 
grades,  and  substantially  better  average  overall  scores  on  the  evaluation. 

The  bombing  analyses-while  less  comprehensive-*also  yielded  good  results.  The 
data  set  was  built  up  from  over  2,500  bombing  runs  of  four  kinds  over  a  nine  month  period 
for  a  medium  attack  squadron.  Average  bombing  accuracy  in  a  month  was  examined  as  a 
function  of  the  amount  of  time  spent  practicing  bombing  in  the  previous  month.  The 
results  indicate  that  a  2-percent  reduction  in  flying  hours  spent  practicing  bombing  is 
associated  with  a  1-percent  increase  in  average  miss  distance  for  this  squadron. 

b.  Unit  training  and  ship  performance.  The  Operational  Propulsion  Plant 
Examination  (OPPE)  provides  an  objective  measure  of  ship  performance  in  the  area  of 
mobility,  a  primary  mission  area  for  all  ships.  It  is  administered  to  each  ship  once  every  18 
months  by  a  team  of  outside  observers,  the  Propulsion  Examining  Board.  CNA  recently 
examined  the  results  of  22  OPPEs  to  see  whether  steaming  provides  quantifiable  training 
benefits  for  a  ship's  crew  (31.  Ships  with  less  than  24  days  of  steaming  in  the  quarter 
before  the  OPPE  had  a  22  percent  chance  of  scoring  average  or  above  on  the  exam.  Ships 
with  more  than  24  days  of  ster-ning  had  a  46  percent  chance  of  scoring  average  or  above. 
It  appears  that  the  intensity  of  training  made  a  quantifiable  difference. 

c.  Personnel  factors  and  shin  performance.  The  earliest  work  seeking  to 
relate  crew  characteristics  to  ship  performance  was  the  Navy  Readiness  Analysis  System 
Methodology  Study  [4]  which  was  completed  in  1969.  It  relied  on  data  from  Operational 
Readiness  Inspections  (ORIs)  performed  at  Fleet  Training  Group  in  Guantanamo  Bay, 
Cuba  for  82  destroyers.  Analyses  were  carried  out  for  21  subsections  of  the  ORI.  For 
each  subsection,  hypotheses  concerning  the  number  of  relevant  personnel  on  board,  how 
long  they  had  been  on  board,  the  presence  of  certain  specialized  skills,  and  the  number  of 
weeks  of  training  received  were  examined.  The  type  of  equipment  aboard  individual  ships 
and  the  availability  of  supplies  were  also  considered.  The  conclusion  of  the  work  was  that, 
"Certain  personnel,  training,  equipment  and  supply  resource  measures  are  substantially 
associated  with  performance  on  the  ORI  subsections.  In  some  cases,  higher  resource 
levels  were  not  associated  with  increases  in  performance.  Most  of  the  relationships  appear 
to  be  logical,  but  a  few  are  hard  to  explain  and  must  be  interpreted  with  caution  until  more 
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is  known  about  the  underlying  processes  involved."  After  almost  seventeen  years,  this 
remains  the  most  wide-ranging  study  of  the  relationships  between  MPT  factors  and  the 
operational  performance  of  ships. 

d.  Training  and  pilot  proficiency  In  the  Air  Force.  A  recent  Air  Force 
study  [5]  investigated  the  relationship  between  flying  hours  and  pilot  proficiency. 
Weapons  delivery  data  from  an  active  duty  A- 10  squadron  and  an  Air  National  Guard  A- 10 
squadron  were  analyzed.  Both  long-term  and  recent  pilot  training  experience  were 
examined  as  potential  determinants  of  bombing  accuracy.  Total  mission  time  by  the  pilot 
was  found  to  have  the  highest  correlation  with  performance.  A  measurable  increase  in 
capability  appeared  to  occur  after  approximately  1,200  hours  of  mission  flying  time  in  the 
A- 10.  No  correlation  was  observed  between  bombing  accuracy  and  either  monthly  flying 
activity  or  the  number  of  events  accomplished  by  pilots  with  low  levels  of  experience. 

While  this  work  is  not  fully  consistent  with  either  the  Navy  work  cited  above  [2]  or 
with  some  unpublished  Air  Force  work  that  covers  additional  types  of  aircraft,  it  would 
have  strong  implications  for  the  personnel  management  policies  applied  to  the  pilot 
community  if  it  were  confirmed  by  further  analysis.  In  particular,  it  would  seem  to  argue 
for  keeping  pilots  in  flying  billets  longer  into  their  careers.  This  would  have  the  additional 
benefit  of  permitting  a  reduction  in  the  number  of  new  pilots  who  have  to  be  trained  every 
year. 

e.  Mental  groan  and  the  performance  of  Army  tank  crews.  As  part  of 
its  efforts  to  determine  the  appropriate  quality  mix  for  Army  personnel,  the  Army 
performed  a  statistical  analysis  of  the  relationship  between  the  individual  characteristics  of 
tank-crew  members  and  tank-crew  range  firing  performance  [6],  Performance  was  scored 
by  Corps  teams  at  the  Grafenwoehr  Training  Area  in  Germany  as  part  of  the  external 
evaluation  portion  of  the  Army  Training  and  Evaluation  Program  (ARTEP).  The  analysis 
covered  both  M60  and  Ml  tanks.  The  mental  group  of  both  the  gunner  and  the  tank 
commander  was  found  to  be  significantly  related  to  performance  for  both  kinds  of  tanks, 
but  less  so  for  the  newer  Ml.  The  Army  used  this  study  to  support  its  request  for  high 
quality  manpower  in  its  submission  to  Congress  in  May  1985. 

f.  Personnel  characteristics  and  training  readiness  in  the  Army 
National  Guard.  The  standard  measure  of  training  readiness  in  the  active  Army  comes 
from  the  UNITREP  system.  It  is  widely  felt  to  be  highly  subjective.  In  the  Army 
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National  Guard,  however,  the  number  of  weeks  to  C-l  training  readiness  is  derived  from  a 
fairly  objective  standardized  evaluation  of  training  readiness  performed  by  an  active  Army 
evaluation  team  observing  the  annual  training  performance  of  units.  An  analysis  by  Rand 
[7]  sought  to  relate  these  evaluations  to  a  wide  range  of  personnel  factors  for  both  officers 
and  enlisted  personnel.  Many  factors  proved  to  be  both  highly  significant  in  a  statistical 
sense  and  also  impoitantquandtatively.  These  included  actual  manning  relative  to  the  level 
authorized,  the  seniority  of  both  officers  and  enlisted  personnel,  educational  level,  unit 
stability,  matches  between  actual  and  required  occupational  specialties,  and  the  marital 
status  and  sex  of  enlisted  personnel  (married  individuals  and  men  performed  better,  other 
things  equal).  While  it  would  probably  have  been  a  good  idea  to  allow  the  determinants  of 
performance  to  vary  according  to  the  kind  of  unit  being  observed,  the  results  obtained  were 
strong  enough  to  be  considered  valid. 

2.  The  Determinants  of  Maintenance  Performance 

Showing  a  connection  of  MPT  factors  to  operational  performance,  as  the  studies  discussed 
above  were  able  to  do,  is  very  appealing.  It  has  two  drawbacks,  however.  First,  it 
depends  on  using  performance  information  that  it  usually  not  centrally  available;  although 
much  of  the  relevant  data  does  routinely  exist  somewhere  in  the  field.  Second,  it  does  not 
lend  itself  to  capturing  the  contribution  to  performance  of  maintenance  personnel.  It  is  true 
that  equipment  must  be  working  before  it  can  be  used  effectively,  but  most  of  the  data  on 
operational  performance  in  the  analyses  cited  above  was  developed  from  the  use  of 
equipment  that  was  known  to  have  a  very  high  probability  of  working  properly.  Both  of 
these  problems  can  be  overcome  by  performing  analyses  of  the  material  readiness  of 
equipment,  a  measure  of  the  performance  of  maintenance  personnel.  Of  course, 
maintenance  personnel  cannot  keep  equipment  in  good  condition  without  proper  technical 
manuals,  test  equipment  and  spare  parts,  but  given  the  environment  in  which  the  personnel 
are  working,  their  skill  is  likely  to  be  the  key  determinant  of  material  readiness.  Studies 
have  successfully  used  generally  available  data  to  relate  the  characteristics  of  enlisted 


maintenance  personnel  to  the  readiness  of  Navy  equipment  Two  of  them  are  discussed 
below.2 

a.  Maintenance  performance  aboard  surface  combatants.  A  study 
completed  at  CNA  in  1977  [9]  looked  at  the  amount  of  down  time  associated  with 
CASREPs  suffered  by  equipment  maintained  by  men  in  each  of  six  maintenance  ratings. 
A  sample  of  91  cruisers,  destroyers  and  frigates  was  analyzed.  The  enlisted  manning 
characteristics  examined  were  crew  size,  education,  entry  test  scores,  training,  length  of 
service,  pay  grade,  turnover,  race  and  marital  status.  Non-personnel  factors  were  included 
in  the  analysis  in  order  to  hold  them  constant  The  determinants  of  material  readiness  were 
found  to  vary  substantially  across  both  ratings  and  the  complexity  of  the  equipment  being 
maintained.  The  most  consistent  result  regarding  personnel  characteristics  was  the 
importance  of  having  senior  people.  In  all  six  ratings,  a  higher  fraction  of  personnel  in 
higher  pay  grades  was  associated  with  less  CASREP  down  time.  This  effect  was 
particularly  pronounced  when  complex  equipment  was  dealt  with.  These  results  would 
seem  to  have  implications  for  policy  regarding  the  proper  mix  of  experience  in  the  Navy, 
for  the  level  of  reenlistment  bonuses,  and  for  assignment  policy. 

b.  Maintenance  performance  in-naval  aYiation.  a  later  cna  study  [10] 
examined  the  material  readiness  of  A-7  squadrons  aboard  aircraft  carriers.  Information  was 
gathered  on  both  the  mission  capable  (MC)  rate  of  the  squadrons  and  on  the  number  of 
sorties  they  were  able  to  fly.  Observations  were  obtained  for  292  quarters  of  squadron 
operations.  Both  of  the  measures  of  performance  were  examined  as  a  function  of  the 
distribution  of  pay  grades  among  squadron  maintenance  personnel.  Squadrons  with  a 
higher  proportion  of  more  senior  personnel  tended  to  do  substantially  better.  Additional 
personnel  in  pay  grades  E-7  and  above  made  a  particularly  large  contribution  to 
performance.  It  was  estimated  that  moving  to  a  more  senior  pay  grade  mix  could  achieve 
the  level  of  performance  observed  in  the  sample  with  18  percent  fewer  personnel,  and  at 
lower  cost. 


2The  studies  cited  here  did  not  use  or  require  information  on  the  characteristics  of  people  who  performed 
particular  maintenance  work.  An  IDA  paper  [8]  notes  that  there  are  analyses  that  could  benefit  from  such 
information. 


3.  The  Determinants  of  Individual-Performance 


It  is  clearly  better  to  be  able  to  relate  MPT  factors  to  the  performance  of  units  than  to 
the  performance  of  individuals.  After  all,  if  a  unit  performs  poorly,  it  doesn't  really  matter 
how  proficient  some  individuals  are  at  their  tasks.  Our  goal  is  to  tie  MPT  policies  to 
outputs  that  really  matter.  However,  it  will  not  always  be  possible  to  quantify  the  impact  of 
MPT  policies  on  either  the  operational  or  maintenance  performance  of  units.  Sometimes, 
as  in  the  case  of  administrative  personnel,  little  direct  effect  on  measured  unit  performance 
can  be  expected.  Sometimes  analysis  of  unit  performance  will  just  not  give  usable  results. 
Many  non-personnel-related  factors  influence  unit  performance,  and  it  will  not  always  be 
possible  to  disentangle  the  impact  of  personnel.  In  such  cases,  an  attempt  should  be  made 
to  judge  the  success  of  MPT  policies  with  reference  to  the  performance  of  individuals--in 
the  actual  work  environment,  if  possible;  in  a  more  artificial  setting,  if  necessary.  Most  of 
the  analyses  relating  MPT  factors  to  performance  have  been  done  ai  the  individual  level.  A 
review  of  a  sample  of  this  work  should  serve  to  demonstrate  the  broad  applicability  of  this 
approach. 

a.  Analysis  of  supervisor  evaluations.  Supervisor  evaluations,  although 
subjective,  provide  a  summary  measure  of  how  well  individuals  perform  in  their  work 
environment  A  1974  survey  asked  supervisors  to  assess  the  net  productivity  of  individual 
first  termers,  relative  to  typical  fully-trained  personnel  with  four  years  of  experience,  at 
various  times  during  their  term  of  service.  Net  productivity  means  how  much  someone 
contributes  to  the  work  group  when  the  amount  of  supervisor  time  lost  to  other  endeavors 
is  considered.  Over  7,000  responses  were  analyzed  for  the  Navy  [11,12].  Although  there 
were  some  anomalous  findings,  the  results  were  generally  supportive  of  the  notion  that 
individuals  with  higher  entry  test  scores  and  more  pre-service  education  are  significantly 
better  performers.  A  related  paper  [13]  estimates  the  time  path  of  productivity  growth 
within  the  first  term.  The  implications  of  this  work  were  examined  [14]  to  gain  insight  into 
the  proper  mix  of  first-termers  and  careerists.  A  program  of  higher  reenlistment  bonuses 
was  recommended. 

b.  Performance  of  Navv  sensor  operators  in  flight  simulators.  Recent 
work  at  CNA  [15]  used  simulator  exercise  results  for  nine  P-3  long-range  ASW  patrol 
squadrons.  The  squadrons  were  all  in  the  Selected  Reserve.  The  sample  included  over 
1,000  individual  exercise  grades.  The  grades  were  developed  from  a  review  of  tapes  made 


during  the  exercises.  The  performance  of  sensor  operators  was  analyzed  as  a  function  of 
education,  entry  test  scores,  pay  grade,  number  of  years  spent  on  active  duty,  the  number 
of  previous  recorded  simulator  trials,  and  whether  the  individual  was  a  TAR  (full  time  on 
active  duty).  The  most  important  predictors  of  performance  were  total  time  spent  on  active 
duty  (more  was  better),  experience  in  simulator  exercises,  and  TAR  status  (TARs  did 
worse,  other  things  equal).  The  effect  of  pay  grade  was  statistically  significant,  but 
quantitatively  of  little  consequence. 

c.  Performance  on  advancement  exams  in  the  Navv.  An  early  study  [16] 
of  the  relative  costs  of  formal  and  on-the-job  training  in  the  Navy  used  attainment  of  a 
passing  score  on  the  E-4  advancement  exam  as  its  indicator  of  successful  completion  of 
training.  It  had  to  take  into  account  the  fact  that  individuals  who  had  received  formal  initial 
skill  training  had  higher  entry-test  scores  than  those  who  had  not  It  estimated  relationships 
between  performance  on  the  advancement  examination  and  scores  on  portions  of  the  battery 
of  entry  tests  taken  by  recruits,  by  rating,  for  both  school  attendees  and  non-attendees. 
These  relationships  were  used  to  estimate  how  the  non-attendees  would  have  fared  on  the 
exams  if  they  had  attended  school,  given  their  entry  test  scores.  Although  the  study  report 
does  not  explicitly  display  the  results  of  the  estimation  procedure, the  entry  test  scores  did 
tend  to  be  significant  predictors  of  performance  on  the  advancement  exams. 

d.  Performance  on  Skill  Qualification  Tests  in  the  Army.  As  was  noted 
earlier,  SQTs  contain  hands-on  and  paper-and-pencil  components.  They  are  mostly 
objective,  but  include  a  subjective  supervisor  s  assessment.  They  may  present  a  somewhat 
biased  view  of  the  level  of  individual  proficiency  because  the  tasks  on  the  tests  can  be 
selectively  prepared  for,  but  the  results  have  been  found  to  behave  similarly  to  indicators  of 
individual  skill  that  do  not  have  this  drawback.  A  Rand  study  [IT]  looked  at  the  SQT 
performance  of  infantrymen  as  a  function  of  scores  on  portions  of  the  ASVAB  (Armed 
Services  Vocational  Aptitude  Battery)  taken  by  new  entrants  into  the  services.  The  entry 
test  scores  were  very  strong  predictors  of  SQT  performance.  The  level  of  pre-service 
educational  attainment  also  played  a  role,  but  it  was  not  as  important.  A  follow-on  study 
[16]  extended  the  analysis  to  four  Army  jobs  and  investigated  the  implications  of  the  results 
for  setting  recruiting  standards  and  determining  job  assignments.  An  Army  Research 
Institute  study  [17]  of  seventeen  MOSs  also  confirmed  the  existence  of  highly  significant 
correlations  between  ASVAB  and  SQT  scores. 


e.  Analysis  of  other  indicators  of  hands.on  individual  performance 
in  the  Army.  The  Systems  Analysis  Agency  of  the  Army’s  Training  and  Doctrine 
Command  (TRASANA)  designed  a  variety  of  MOS-specific  hands-on  tests  that  were 
administered  under  controlled  conditions  by  TRASANA  personnel  to  individuals  serving 
with  Army  units.  Performance  was  studied  as  a  function  of  the  Armed  Forces  Qualification 
Test  (AFQT)  score,  education  level,  sex,  pay  grade,  time  in  service,  and  time  in  MOS  [20]. 
Soldiers  in  AFQT  categories  in  I-HIA  performed  approximately  10  percent  better  than  HIB 
soldiers.  AFQT  was  a  more  important  influence  in  virtually  all  instances  than  either 
education  or  experience.  This  finding,  linking  AFQT  scores  with  individual  hands-on 
performance,  was  consistent  with  a  1969  analysis  performed  by  the  Human  Resources 
Research  Office  [21]. 

4.  The  State  of  the  Literature  on  People  and  Performance 

For  both  the  Army  and  the  Navy,  modest  bodies  of  literature  exist  that  link  the 
characteristics  and  experience  of  personnel  with  performance.  The  work  for  both  services 
gives  broad  but  shallow  coverage  to  a  range  of  occupations.  The  Navy  analyses  have  the 
virtue  of  being  more  oriented  toward  the  performance  of  units  rather  than  individuals.  The 
Army  results  tend  to  highlight  the  performance  of  entry-test  scores  as  predictors  of 
performance.  The  Navy  results  give  more  prominence  to  experience.  Not  surprisingly,  the 
predictors  of  performance  vary  substantially  with  the  task  being  performed.  While  some 
valuable  policy  insights  can  be  gained  from  this  literature,  many  key  issues  remain  very 
much  in  need  on  further  analysis.  Prominent  among  them  are  the  appropriate  level  of 
accession  quality  in  the  Navy  and  the  relative  efficacy  of  different  approaches  to  training. 

E.  CONCLUSIONS  AND  RECOMMENDATIONS 

Conceptual  links  exist  between  MPT  policy  and  the  kinds  of  outputs  the  country 
seeks  to  buy  from  the  Defense  Department  as  a  whole-military  performance,  readiness  and 
defense  capability.  The  use  of  explicit  quantitative  links  to  such  outputs  could  lead  to  the 
development  of  improved  MPT  policy. 

Failure  to  undertake  the  development  of  quantitative  relationships  between  MPT 
policy  and  aspects  of  military  performance  has  traditionally  been  blamed  on  the  lack  of 
performance  data.  But  plenty  of  performance  data  exist  for  the  Services  in  general  and  for 
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the  Navy  in  particular.  Information  on  the  operational  and  maintenance  performance  of 
units,  as  well  as  information  on  the  performance  of  individuals  is  gathered  routinely. 
Indicators  of  operational  proficiency  are  generally  not  available  from  any  central  repository, 
but  indicators  of  maintenance  and  individual  proficiency  are  centrally  available. 

Even  though  building  relationships  between  people  and  performance  is  a  relatively 
under-researched  area,  over  the  years  a  modest  literature  has  developed  on  the  topic.  Over 
fifteen  studies  have  been  identified  that  quantify  links  between  MPT  factors  and 
performance.  Taken  together  they  address  the  payoff  to  training,  experience,  mental  ability 
and  education,  among  other  things.  They  bear  on  the  Navy,  Army  and  Air  Force.  They 
examine  reserve  as  well  as  active  forces.  All  rely  on  statistical  techniques  to  establish 
approximate  relationships.  This  body  of  literature  demonstrates  conclusively  that  the 
information  needed  to  begin  to  manage  MPT  polices  with  an  eye  toward  military 
performance  can  be  obtained. 

The  development  and  use  of  links  between  MPT  factors  and  performance  should 
become  a  central  focus  of  the  Navy's  MPT  research.  While  this  could  be  accomplished  in  a 
decentralized  fashion,  establishment  of  a  separate  identifiable  program  to  spearhead  this 
development  is  worth  considering.  Such  a  program  would  minimize  the  problem  of  wasted 
duplication  of  effort  and  would  serve  as  a  storehouse  of  institutional  knowledge. 

However  the  effort  is  organized,  it  ought  to  proceed  along  several  tracks 
simultaneously.  Analyses  should  be  initiated  using  information  on  both  maintenance  and 
individual  performance  for  as  wide  a  range  of  ratings  as  possible.  Whatever  operational 
performance  data  can  be  obtained  should  also  be  pressed  into  service.  At  the  same  time,  a 
centralized  data  base  on  operational  performance  should  be  developed.  Research  should 
both  try  to  quantify  relationships  involving  the  determinants  of  performance  and  look  at  the 
policy  implications  of  known  or  hypothesized  links  between  MPT  factors  and  performance. 
It  is  important  to  determine  the  extent  to  which  giving  a  performance  orientation  to  MPT 
policy  making  could  make  a  difference.  Harking  back  to  the  policy  areas  identified  at  the 
beginning  of  this  paper-requirements  determination,  compensation,  retirement  and 
training-the  following  specific  projects  could  be  undertaken. 

•  Requirements  determination.  Use  data  on  the  operational  performance  of  ships  to 
study  the  value  of  having  more  individuals  in  higher  mental  groups  and  the  value 
of  having  more  experienced  personnel,  for  a  variety  of  operating  ratings. 
Individual  performance  data  should  be  used  if  operational  performance  data  is  not 
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available.  Perform  a  similar  analysis  using  information  on  material  readiness 
(probably  "4855"  data)  for  individuals  in  selected  maintenance  ratings.  Existing 
analyses  have  identified  these  as  areas  with  potentially  high  payoffs. 

•  Compensation.  Build  a  model  for  developing  efficient  compensation  packages 
from  information  on  the  relationships  between  experience  and  productivity. 
Apply  the  model  to  die  results  of  existing  studies,  such  as  [10].  This  could  give 
some  insight  into  whether  today's  pay  profile  makes  sense. 

•  Retirement  Analyze  the  effectiveness,  and  cost,  of  various  retirement  packages 
for  meeting  alternative  sets  of  requirements  for  people  by  length  of  service. 
These  first  three  studies  could  indicate  ways  in  which  requirements, 
compensation  and  retirement  might  be  changed  in  mutually  supportive  ways. 

•  Training.  Study  pilot  proficiency  as  a  function  of  both  recent  training  history  and 
accumulated  flying  time,  as  well  as  the  personal  characteristics  of  pilots.  This 
work  could  have  implications  for  the  selection,  compensation  and  career 
management  of  pilots,  and  for  management  of  the  flying-hour  program. 

•  Data  development  Initiate  construction  of  a  central  repository  of  information  on 
the  operational  performance  of  units.  This  should  encompass  the  results  of 
training  inspections,  selected  exercises  and  excellence  award  competitions,  as 
well  as  information  on  bombing  accuracy  and  performance  on  TACTS  ranges. 
Since  the  purpose  of  developing  this  data  base  is  to  build  generally  applicable 
relationships,  not  to  evaluate  individuals  or  units,  a  strong  commitment  should  be 
made  to  eschew  its  use  for  such  evaluation  purposes.  This  commitment  should 
include  destroying  identifying  codes  as  quickly  as  is  feasible. 

•  Long-term  research  planning.  Just  as  a  ship  cannot  be  turned  on  a  dime,  it  will 
take  time  for  the  Navy  to  move  to  a  performance  orientation  in  MPT  management 
Developing  a  research  strategy  to  make  the  change  as  quickly  as  possible  could  be 
most  fruitful. 

This  program  strives  for  balance  between  studying  the  determinants  of  performance 
and  learning  to  use  the  inf  rmation  from  such  studies,  as  well  as  balance  between 
developing  insights  quickly  and  building  for  the  long  run.  Its  adoption  would  imply  a 
commitment  to  seriously  consider  changing  the  orientation  of  MPT  policy-making  in  the 
Navy,  to  make  it  steer  by  the  star  of  improved  performance.  Substantial  benefits  can  be 
foreseen.  The  rather  sparse  analysis  done  to  date  already  builds  a  strong  case  for  greater 
experience  in  the  Navy's  enlisted  force-and  for  the  level  of  bonuses  necessary  to  get  it.  As 
additional  research  is  performed,  the  Navy  would  increasingly  be  in  a  position  to  focus  on 
getting  the  most  for  its  MPT  dollars. 
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